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Summary rel 2011-No. 103 

Dropout prevention programs in nine 
Mid-Atlantic Region school districts: 
additions to a dropout prevention database 



This report describes dropout prevention 
programs identified by respondents in 
nine school districts in the Mid-Atlantic 
Region, along with a searchable database 
of the programs. The programs expand a 
database developed in an earlier North- 
east and Islands Region study. Only 1 
of the 58 identified programs has been 
reviewed for effectiveness by the What 
Works Clearinghouse. 

Dropping out of high school is a serious con- 
cern. It is associated with numerous harmful 
effects for dropouts and their communities. 
There is evidence that some dropout preven- 
tion programs and policies are effective, but 
those in use in the Mid-Atlantic Region have 
not been documented. 

The current study replicates work of Regional 
Educational Laboratory (REL) Northeast 
and Islands. It describes dropout prevention 
programs in nine Mid- Atlantic Region (Dela- 
ware, the District of Columbia, Maryland, 

New Jersey, and Pennsylvania) school dis- 
tricts serving communities with populations 
of 24,742-107,250 (as of July 2008). All nine 
districts have high dropout rates, large racial/ 
ethnic minority student populations, and high 
percentages of students from households living 
below the poverty line. The study is driven by 
two research questions: 



• What are the characteristics of dropout 
prevention programs and policies in the 
nine districts? 

• Which programs have been reviewed by 
the U.S. Department of Education’s What 
Works Clearinghouse, and what were its 
findings? 

The study found that: 

• No district reported dropout prevention 
policies apart from those establishing the 
reported programs. 

• Only one program model was reported by 
more than one district. Because the pro- 
gram was funded and implemented differ- 
ently in the two districts that reported it, it 
is treated as two programs. 

• The most common core strategies were 
advocating for student needs (64 percent 
of programs), engaging and supporting 
families (57 percent), and monitoring 
school attendance (53 percent). 

• The most common service goals were to 
improve academic performance (95 percent 
of programs), decrease truancy (66 per- 
cent), and provide support during transi- 
tions (60 percent). 



SUMMARY 



• The most common student subgroups tar- 
geted were students with academic needs 
(90 percent of programs), students from 
low socioeconomic status families (60 
percent), and special needs students with 
behavioral challenges (57 percent). 

• Programs that targeted specific grades 
were most likely to focus on students in 
grades 9 or 12. 

• Teachers were involved in 86 percent of 
reported programs, guidance counselors 
in 78 percent, and principals or other 
administrators in 67 percent. 

• The most common forms of community 
involvement engaged parents (69 percent 
of programs), youth or social services staff 
(28 percent), mental health services staff 
(28 percent), police (22 percent), and men- 
toring program staff (21 percent). Twelve 
programs (21 percent) reported no com- 
munity involvement. 

• Districts funded all or part of 79 percent 
of reported programs; state governments 
had some financial role in 41 percent, the 
federal government in 26 percent, and pri- 
vate sources in 7 percent. Four programs 
(7 percent) did not report a funding source. 



• As of May 1, 2010, only 1 of the 58 
programs — Talent Development High 
Schools — had been reviewed by the What 
Works Clearinghouse, which found only 
one small study that met its evidence stan- 
dards with reservations. 

The results of this study were added to the REL 
Northeast and Islands database of dropout 
prevention programs. The database documents 
local use of dropout prevention programs and 
indicates which of the reported programs have 
been reviewed by the U.S. Department of Edu- 
cation’s What Works Clearinghouse. It enables 
school administrators to identify districts with 
programs like their own, programs targeting 
specific student groups, and programs employ- 
ing core strategies they might want to investi- 
gate. The database provides information that 
could lead to cross-district collaboration in 
grant seeking or on joint programs with insti- 
tutions of higher education and other regional 
institutions and agencies. It will be updated 
and expanded periodically and can be ac- 
cessed at www.relnei.org/research.educational. 
dropoutdb.php. See box 2 in the main report 
for a basic guide to using the database and 
appendix A in Myint-U et al. (2008, revised 
2009) for the complete users guide. 

February 2011 
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This report 
describes dropout 
prevention 
programs identified 
by respondents in 
nine school districts 
in the Mid-Atlantic 
Region, along 
with a searchable 
database of the 
programs. The 
programs expand 
a database 
developed in an 
earlier Northeast 
and Islands Region 
study. Only 1 of 
the 58 identified 
programs has 
been reviewed for 
effectiveness by 
the What Works 
Clearinghouse. 



WHY THIS STUDY? 

Dropping out of high school is a serious concern. It 
is associated with numerous adverse outcomes for 
dropouts and their communities. There is evidence 
that some dropout prevention programs and poli- 
cies have positive effects, but those in use in the 
Mid- Atlantic Region have not been documented. 

This report describes dropout prevention pro- 
grams in nine Mid- Atlantic Region (Delaware, the 
District of Columbia, Maryland, New Jersey, and 
Pennsylvania) school districts and a searchable 
database of such programs. The database supports 
cross-district collaboration involving dropout 
prevention strategies and documents local use 
of programs and strategies reviewed by the U.S. 
Department of Education’s What Works Clearing- 
house for evidence of effectiveness. Both the study 
and the database replicate work of Regional Edu- 
cational Laboratory (REE) Northeast and Islands 
(Myint-U et al, 2008; revised 2009). 



Regional need 

The REE Northeast and Islands study prompted 
requests for similar research from state educa- 
tion agency staff in Delaware and New Jersey and 
from members of the REE Mid-Atlantic Governing 
Board at a meeting in February 2008. To repli- 
cate the REE Northeast and Islands study in the 
Mid-Atlantic Region, nine school districts with 
high dropout rates, high percentages of racial/ 
ethnic minority students, and high percentages of 
students from households living below the poverty 
line were selected for examination. The districts 
served communities with populations of 24,742- 
107,250 (as of July 2008; see box 1 and appendix A 
for details of the study methodology). 

Two research questions drive the report: 

• What are the characteristics of dropout preven- 
tion programs and policies in the nine districts? 

• Which programs have been reviewed by the 
U.S. Department of Education’s What Works 
Clearinghouse? 
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DROPOUT PREVENTION PROGRAMS IN NINE MID-ATLANTIC REGION SCHOOL DISTRICTS 



BOX 1 

Study methodology 

This report replicated the Regional 
Educational Laboratory (REL) North- 
east and Islands study in sample 
selection and data collection (Myint- 
U et al. 2008, revised 2009). 

Selecting the sample. A list of dis- 
tricts serving mid-size cities in each 
Mid-Atlantic Region state was com- 
piled using the National Center for 
Education Statistics (NCES) defini- 
tion of mid-size city (a population of 
less than 250,000; National Center 
for Education Statistics 2008). The 
districts were merged into a single 
list and ranked from high to low by 
cumulative four-year dropout rate, 
percentage of racial/ethnic minority 
students, and percentage of students 
from households living below the 
poverty line. 

The rankings for each factor were 
summed to form a cumulative 
rank and the nine highest ranking 
districts were selected. Three of the 
nine failed to respond to invitations 
to participate in the project and were 
replaced by the next three highest 
ranking districts. 

Collecting the data. Supporting 
materials, based on materials used 
in the REL Northeast and Islands 
study, included introductory letters to 
school personnel (appendix C), model 
commitment letters for district use 



(appendix D), interview guides (ap- 
pendix E), and a template for record- 
ing information (appendix E). The 
template provided lists of response 
categories for each question, so quali- 
tative data did not have to be coded. 

Next, interview questions were devel- 
oped on programs and policies focus- 
ing on dropout prevention, dropout 
reduction, school completion, and 
students at a higher risk of dropping 
out. The NCES event dropout rate was 
used to define dropout: “students of 
any grade who leave school between 
the beginning of one school year and 
the beginning of the next without 
earning a high school diploma or its 
equivalent” (Laird, DeBell, and Chap- 
man 2006). 

Eollowing an Internet search for 
preliminary information about local 
dropout problems and what districts 
and states were doing to address 
them, district administrative offices 
were contacted through explor- 
atory phone calls and emails. The 
research team explained the project, 
answered questions, and asked to 
be connected to the person most 
directly responsible for dropout 
prevention efforts. 

Respondents received explanatory 
materials, a model for a commitment 
letter from the superintendent, and 
interview consent forms. Respon- 
dents agreed to secure district com- 
mitment letters, complete consent 



forms, and gather information on 
district dropout prevention programs 
and policies by consulting their files 
and those of colleagues. Commitment 
letters, signed by the superintendent 
or the superintendent’s designee, 
were received for each district, and 
consent forms were received for each 
respondent. 

Respondents then received a tem- 
plate for recording data (see appen- 
dix E ) several days before scheduled 
interviews so they could prepare by 
consulting colleagues or compiling 
information. Respondents were asked 
to review the core strategies de- 
scribed in the template to guide them 
in what programs to report. 

The respondent in each district was 
interviewed by phone for about 15-30 
minutes and asked to send additional 
materials describing the programs. 

After each interview, the research 
team looked for discrepancies and 
contacted respondents for clarifica- 
tion. Each respondent received a 
complete data file, confirmed its 
accuracy, and confirmed that the su- 
perintendent or designee authorized 
its inclusion in the database. 

All project data, including written 
notes and supplemental materials, 
were stored at Rutgers University, in 
accordance with the requirements of 
Rutgers’ Institutional Review Board 
for the Protection of Human Subjects. 



What the literature shows 

Two U.S. Department of Education Institute of 
Education Sciences sources guide educators and 
policymakers seeking effective dropout prevention 
programs and policies: 



• The What Works Clearinghouse evaluates 
middle school, junior high school, high 
school, and community-based dropout pre- 
vention interventions. As of May 1, 2010, it 
had rated 17 interventions and found 12 with 
potentially positive effects (appendix B). 
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• The Dropout Prevention Practice Guide 
(Dynarski et al. 2008) recommends ways for 
educators and policymakers to reduce dropout 
rates. It rates the level of evidence for its six 
recommendations as strong, moderate, or lowd 

• Uses data systems that support a realistic 
diagnosis of the number of students who 
drop out and that help identify students 
at high risk of dropping out (level of 
evidence: low). 

• Assigns adult advocates to students at 
risk of dropping out (moderate). 

• Provides academic support and enrich- 
ment to improve academic performance 
(moderate). 

• Implements programs to improve class- 
room behavior and social skills (low). 

• Personalizes the learning environment 
and instructional process (moderate). 

• Provides rigorous instruction to en- 
gage students in learning and transfer 
the skills students need to graduate 
and to succeed after they leave school 
(moderate). 

The literature on the harmful outcomes associated 
with dropping out is rich. Peer-reviewed articles in 
Belfield and Levin (2007) report that the earnings 
of high school dropouts are far less than those of 
nondropouts (Rouse 2007); female dropouts are 
far more likely to be on welfare than women with a 
high school diploma or more (Waldfogel, Garfin- 
kel, and Kelly 2007); high school graduates live 
6-9 years longer than high school dropouts and 
are less likely to become ill or disabled (Muennig 
2007); and dropouts are more likely to commit 
violent crimes and be imprisoned at some point in 
their lives (Moretti 2007). 



students and society. 

Rouse (2007) shows that 
high school graduates 
contribute far more tax 
revenue than do high 
school dropouts. Muen- 
nig (2007) estimates that 
each additional high 
school graduate could 
save the government 

$39,000 in health care costs (in present value 
terms) over a lifetime from age 20. Moretti (2007) 
explains how increasing the number of high 
school graduates could significantly reduce crime 
related costs and add billions in wage dollars to 
the economy. 



In the Mid-Atlantic 
Region, as in the 
United States, Black 
and Hispanic students 
(and American Indian 
students in Maryland and 
Pennsylvania) have the 
lowest graduation rates 



The extent of dropout problems can be seen in 
average freshman graduation rates (the percent- 
age of an entering freshman class that graduates 
within four years) at both national and state levels. 
Stillwell (2009) reports a national public school 
graduation rate of 73.9 percent for 2006-07 by 
dividing the number of 2006/07 diploma recipi- 
ents by the average number of 2002/03 grade 8 
students, 2003/04 grade 9 students, and 2004/05 
grade 10 students. In the Mid-Atlantic Region, as 
in the United States, Black and Hispanic students 
(and American Indian students in Maryland and 
Pennsylvania) have the lowest graduation rates 
(table 1). 



Graduation gaps across gender and racial/ethnic cat- 
egories can also be seen in status completion rates — 
the high school graduation status of a particular age 
group. So Cataldi, Laird, and KewalRamani (2009) 
calculated the national percentage of 18- through 
24-year-olds in 2007 who were not enrolled in high 
school but who had earned a high school diploma or 
equivalent credential, such as a General Educational 
Development (GED) certificate. They report that: 

• Most (89.0 percent) had received a high school 
diploma or equivalent credential. 



Belfield and Levin (2007) argue that increasing 
high school graduation rates can benefit both 



Women had a higher status completion rate 
(90.6 percent) than men (87.4 percent). 
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DROPOUT PREVENTION PROGRAMS IN NINE MID-ATLANTIC REGION SCHOOL DISTRICTS 



TABLE 1 

Average high school freshman graduation rates in Mid-Atlantic Region jurisdictions, by race/ethnicity, 2006/07 



Race/ethnidty 


Delaware 


District of 
Columbia 


Maryland 


New Jersey 


Pennsylvania 


All students 


71.9 


54.9 


80.0 


84.4 


83.0 


American Indian 


77.1 


— 


71.6 


91.2 


64.1 


Asian 


100.0 


— 


99.6 


100.0 


98.7 


Hispanic 


64.7 


— 


76.8 


74.3 


68.4 


Black 


64.0 


— 


72.4 


72.9 


64.6 


White 


75.8 


— 


84.3 


88.4 


87.3 



— is not available. 
Source.- Stillwell 2009. 



• Whites (93.5 percent) had a higher status com- 
pletion rate than Asians (93.1 percent), Blacks 
(88.8 percent), and Hispanics (72.7 percent). 



• Just more than half (56.1 percent) of foreign- 
born Hispanics had completed high school, 
compared with 85.9 percent of “first genera- 
tion” native-born Hispanics and 85.1 percent 
of “second generation or higher” native-born 
Hispanics. 



Participating Mid- 
Atlantic Region districts 
reported a wide range 
of dropout prevention 
programs, but only 
one has been reviewed 
by the What Works 
Clearinghouse 



Students from some racial/ethnic groups and 
students from low-income households are at an 
increased risk of not completing high school. A 
study of national event dropout rates (the percent- 
age of students ages 15-24 who dropped out of 
grades 10-12 in public or private schools) be- 
tween October 2006 and October 2007 found the 
dropout rate for White students to be 2.2 percent, 
compared with 7.5 percent for Asian students, 6.0 
percent for Hispanic students, and 4.5 percent for 
Black students. Students from households with 
incomes in the bottom 20 percent of the sample 
had a dropout rate of 8.8 percent; 
students from households with 
incomes in the top 20 percent 
had a dropout rate of 0.9 percent 
(Cataldi et al. 2009). 



A key to reducing dropout rates is 
understanding why students leave 
school before graduating — by 
identifying factors that are highly 



correlated with dropping out. Neild and Balfanz 
(2006) found that for on-time grade 10 students 
(those who have not repeated a grade), risk factors 
included scoring low in reading, having atten- 
dance below 80 percent, earning fewer than five 
credits, having a baby during the year, and experi- 
encing an out-of-home juvenile justice placement 
during the year. 

Bridgeland, Dilulio, and Morison (2006) sought 
dropouts’ opinions about why they left high school 
without graduating. They reported that 69 percent 
of dropouts said that they were not motivated or 
inspired to work hard; 47 percent said a major 
reason was that their classes were not interesting; 
45 percent said they started high school poorly 
prepared; 35 percent said they were “failing in 
school”; 32 percent cited personal reasons, such as 
needing to get a job, becoming a parent, or hav- 
ing to care for a family member; and 29 percent 
expressed doubts that they could have met their 
school’s requirements for graduation. 



FINDINGS 

The findings draw on district respondents’ views 
of what constitutes a dropout prevention program, 
as shaped by the materials sent by the research 
team before the interviews. Participating Mid- 
Atlantic Region districts reported a wide range of 
dropout prevention programs, but only one has 
been reviewed by the What Works Clearinghouse. 
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What are the characteristics of dropout prevention 
programs and policies in the nine districts? 

Each participating district reported 5-11 dropout 
prevention programs, for a total of 58 programs 
across the nine districts. No district reported 
dropout prevention policies apart from those es- 
tablishing the reported programs. And only once 
did two districts report using the same program 
model — Education Eeading to Employment and 
Career Training. But how the two districts imple- 
mented this program — one as a state-funded 
program involving seven categories of staff and 
collaboration with a community health clinic and 
the other as a privately funded program involv- 
ing only guidance counselors — was so different 
that the programs are treated as distinct in this 
report. 

Core strategies. The most common core strategies 
identified by respondents were advocating for 



student needs (64 percent of programs), engaging 
and supporting families (57 percent), and moni- 
toring attendance (53 percent; table 2). 

Service goals. The most common service goals 
(the means by which the program seeks to help 
students stay in school, progress in school, or com- 
plete school)^ reported were to improve academic 
performance (95 percent of programs), decrease 
truancy (66 percent), and provide support during 
transitions (60 percent; table 3). 

Targeted student subgroups. Students with aca- 
demic needs was the most commonly targeted stu- 
dent subgroup— a focus of 90 percent of programs 
(table 4). Other subgroups targeted by 50 percent 
or more of programs were students from low 
socioeconomic status families (60 percent), special 
needs students with behavioral challenges (57 per- 
cent), and students who are chronically truant or 
absent (53 percent). 



TABLE 2 

Core strategies of 58 dropout prevention programs in nine Mid-Atlantic Region school districts, 2008 



Core strategy 


Number 


Percent 


Advocating for student needs 


37 


64 


Engaging and supporting families 


33 


57 


Monitoring attendance 


31 


53 


Mentoring 


25 


43 


Transforming the school environment 


22 


38 


Case management/service coordination 


21 


36 


Community collaboration 


20 


34 


Career education and workforce readiness 


20 


34 


Social and emotional transition support 


18 


31 


Accelerated credit accumulation 


18 


31 


Tutoring/extra classes 


17 


29 


Individualized or culturally/linguistically relevant instruction 


16 


28 


Instructional technologies 


15 


26 


Systemic/policy renewal 


14 


24 


Out-of-school enhancement 


13 


22 


Social and emotional learning curricula 


9 


16 


Professional development 


4 


7 


Enhancing English language proficiency 


1 


2 



Note: A program can employ more than one strategy. 

Source; Authors' analysis based on data from district reports of core strategies. 
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TABLE 3 

Service goals of 58 dropout prevention programs in nine Mid-Atlantic Region school districts, 2008 



Service goal 




Programs 




Number 




Percent 


Improve academic performance 


55 




95 


Decrease truancy 


38 




66 


Provide support during transitions 


35 




60 


Increase school attachment 


34 




59 


Address behavioral challenges 


29 




50 


Promote college planning and linkages 


27 




47 


Provide career planning and preparation 


27 




47 


Provide mental health support 


19 




33 


Address school safety and environment 


17 




29 



Note: A program can have more than one service goal. 

Source.- Authors' analysis based on data from district reports of service goals. 



TABLET 

Student subgroups targeted by 58 dropout prevention programs in nine Mid-Atlantic Region school 
districts, 2008 



Subgroup 


Number 


Programs 


Percent 


Students with academic needs 


52 




90 


Students from low socioeconomic status families 


35 




60 


Special needs students with behavioral challenges 


33 




57 


Students who are chronically truant or absent 


31 




53 


Special needs students with learning disabilities 


22 




38 


Special needs students with mental health needs 


22 




38 


Pregnant teens/teen mothers 


21 




36 


English language learner students 


20 




34 


Students who would be the first in their family to attend college 


18 




31 


Students returning from incarceration 


16 




28 



Note: A program may have more than one target subgroup. 

Source.- Authors' analysis based on data from district reports of student subgroups targeted. 



Targeted grade levels. Of the 58 reported dropout 
prevention programs, 48 targeted multiple grades: 
12 focused on grades 9-12; 12 on grades 6-12; 5 
on grades 7-12; 5 on grades 6-8; and 5 on grades 
11-12. By contrast, only 10 programs focused on 
students in a single grade: 5 on grade 9; 3 on grade 
12; and 1 each on grades 8 and 11 (table 5). 

District stajf involvement. Programs involved 
a wide variety of staff. The most common were 



teachers (86 percent of programs), guidance 
counselors (78 percent), and principals or other 
administrators (67 percent; table 6). 

Community involvement. Parent involvement was 
the most commonly reported form of community 
involvement; community agency staff played some 
role in many programs (table 7). The most com- 
mon community agencies involved were depart- 
ments of youth or social services (28 percent). 
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TABLE 5 

Grades targeted by 58 dropout prevention 
programs in nine Mid-Atlantic Region school 
districts, 2008 



Grade 


Exclusively 
on grade 


Partially 
on grade 


6 


0 


17 


7 


0 


24 


8 


1 


27 


9 


5 


35 


10 


0 


34 


11 


1 


38 


12 


3 


37 



Source.- Authors' analysis based on data from district reports of grades 
targeted. 



TABLE 7 

Community involvement in 58 dropout 
prevention programs in nine Mid-Atlantic Region 
school districts, 2008 



Position 


Programs 
Number Percent 


Parents 


40 


69 


Departments of youth/social services 


16 


28 


Mental health services 


16 


28 


Police 


13 


22 


Mentoring program 


12 


21 


Truancy court 


11 


19 


Local higher education 


10 


17 


Tutors 


8 


14 


Flealth clinic 


6 


10 


Religious affiliate 


5 


9 


Other volunteers 


5 


9 


AmeriCorps members 


4 


7 


Corporations 


3 


5 


Community-based organization 


1 


2 


No reported community involvement 


12 


21 



/Vote.- A program may have more than one type of community 
involvement. 

Source.- Authors' analysis based on data from district reports of com- 
munity involvement. 



TABLE 6 

District staff involvement in 58 dropout 
prevention programs in nine Mid-Atlantic Region 
school districts, 2008 



Position 


Programs 

Number Percent 


Teacher 


50 


86 


Guidance counselor 


45 


78 


Principal/administrator 


39 


67 


Special education staff 


26 


45 


School nurse 


20 


34 


Truancy/attendance officer 


19 


33 


Paraprofessional 


15 


26 


Psychologist/therapist 


15 


26 


Social worker 


15 


26 


Parent-community liaison 


14 


24 


Adjustment counselor 


11 


19 


Entire school staff 


10 


17 


Crisis counselor 


8 


14 


Dropout specialist 


8 


14 


Tutor 


7 


12 


Director 


7 


12 


Resource officer 


7 


12 


Security guard 


6 


10 


Day care provider 


5 


9 


Support specialist 


5 


9 


Student advocate 


4 


7 


Behavioral resource office staff 


4 


7 


Grant writer 


2 


3 


Grant manager 


1 


2 


Nutrition counselor 


1 


2 



Note: A program may involve more than one type of staff. In some cases, 
districts indicated degrees of involvement by reporting both individual 
staff who had major involvement and "entire school staff" to indicate 
that all staff had some involvement. 

Source: Authors' analysis based on data from district reports of staff 
involvement. 



Twelve programs (21 percent) reported no com- 
munity involvement. 

Funding sources. Districts funded all or part of 
79 percent of programs; state governments were the 
most common providers of outside funds (41 per- 
cent of programs); and no funding sources were 
reported for four programs (7 percent; table 8). 



mental health services (28 percent), police (22 
percent), and mentoring programs (21 percent). 
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TABLE 8 

Funding sources for 58 dropout prevention 
programs in nine Mid-Atlantic Region school 
districts, 2008 



Funding source 


Programs 


Number 


Percent 


District 


46 


79 


State 


24 


41 


Federal 


15 


26 


Private 


4 


7 


None reported 


4 


7 



Note: A program may have more than one funding source; includes 
partial or full funding. 

Source.- Authors' analysis based on data from district reports of funding 
sources. 



Which programs have been reviewed by the 
U.S. Department of Education's What Works 
Clearinghouse, and what were its findings? 

As of May 1, 2010, the What Works Clearinghouse 
website (www.whatworks.ed.gov) had reported on 
the effectiveness of 17 dropout prevention inter- 
ventions, based on reviews of studies meeting its 
standards (see appendix B).^ 

Only one of these interventions — Talent Develop- 
ment High Schools — had been implemented by a 
sample Mid- Atlantic Region district at the time of 
the study. The What Works Clearinghouse (2007, 
p. 1) describes Talent Development High Schools as: 

A school reform model for restructuring 
large high schools with persistent atten- 
dance and discipline problems, poor student 
achievement, and high dropout rates. The 
model includes both structural and cur- 
riculum reforms. It calls for schools to 
reorganize into small “learning communi- 
ties” — including ninth-grade academies for 
first-year students and career academies for 
students in upper grades— to reduce student 
isolation and anonymity. It also emphasizes 
high academic standards and provides all 
students with a college-preparatory aca- 
demic sequence. 



Talent Development High Schools had “poten- 
tially positive effects on progressing in school,” as 
measured by credit accumulation, grade promo- 
tion, or highest grade completed. But the amount 
of evidence was “small” and “no studies [of the 
model] that met the [What Works Clearinghouse] 
evidence standards . . . addressed staying in school 
or completing school” (What Works Clearing- 
house 2007, p. 1). 



ADDING TO THE REGIONAL EDUCATIONAL 
LABORATORY NORTHEAST AND ISLANDS 
DATABASE 

Data from this study were integrated into the 
REL Northeast and Islands database of dropout 
prevention programs. The database documents 
local use of dropout prevention programs and in- 
dicates which of the reported programs have been 
reviewed by the U.S. Department of Education’s 
What Works Clearinghouse. It enables school 
administrators to identify districts with programs 
like their own, programs targeting specific student 
groups, and programs employing core strate- 
gies they might want to investigate. The database 
provides information that could lead to district 
collaboration in grant seeking or on joint pro- 
grams with institutions of higher education and 
other regional institutions and agencies. It will be 
updated and expanded periodically and can be 
accessed at www.relnei.org/research.educational. 
dropoutdb.php. See box 2 for a basic guide to using 
the database and appendix A in Myint-U et al. 
(2008, revised 2009) for the complete users guide. 

This database is maintained by Regional Educa- 
tional Laboratory (REL) Northeast and Islands or 
its designee, but other regional educational labora- 
tories, states, and organizations may be authorized 
to add data, as appropriate. REL Northeast and 
Islands or its designee also makes any decisions 
on updating database fields. REL Northeast and 
Islands and REL Mid-Atlantic are discussing plans 
for collaborative efforts to update and expand 
the database, including the addition of data from 
larger districts and urban districts with lower 
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BOX 2 

Using the dropout prevention 
database 

The database is located at www.relnei. 

org/research.educational.dropoutdb. 

php. 

Design. The database is divided into 
four linked page layouts: 

• Program or Policy Details over- 
views each program or policy in 
the database, including service 
goals, core strategies, staffing, 
and whether the program is 
reviewed by the What Works 
Clearinghouse. 

• District Details overviews each 
participating district and lists 
the programs and policies it 
was implementing in the year 
reported. 

• School Details overviews each 
school in participating districts 
and lists the programs and poli- 
cies it was implementing in the 
year reported; only schools that 
were implementing one or more 



programs are included in the 
database. 

• Programs List View Read-Only 
lists all the districts and pro- 
grams in the database. 

Navigating. When you log in, you will 
see the Program or Policy Details 
page. To move to a different page 
layout, use the Layout pull-down 
menu in the left navigation bar. To 
move to another record within a page 
layout, click on Previous or Next in 
the top right corner of the page. On 
the Program List View Read-Only 
page, arrows on the notebook icon in 
the left navigation bar let you scroll 
through the entire list, 25 records at 
a time. Click the arrows to the left of 
“District” to sort districts in ascend- 
ing or descending order or the arrows 
to the left of “Program Name” to 
sort program names in ascending or 
descending order. 

Searching. Searches using customized 
criteria can be conducted in all four 
page layouts. To search for records 
meeting specific criteria on the 
Program or Policy Details page, select 



Show All Records (the eye icon under 
Browse in the left navigation bar) 
and then the Find button (the blue 
circle with a magnifying glass at the 
top middle of the page). This opens a 
blank Program or Policy Details form. 
To search by program name, enter all 
or some of the program name into the 
Name field and click Perform Find 
on the left navigation bar. To search 
by criteria, select one or more criteria 
for which you would like to find a 
matching program or policy and click 
Perform Find. To search for pro- 
grams reviewed by the What Works 
Clearinghouse, for example, select Yes 
in that field and click Perform Find. 
The database will find all records that 
match your selected criteria. To see 
matching programs in list view, select 
Program List View under Layout. 

To go back to accessing all records, 
select Show All Records. The District 
Details and School Details pages can 
be searched in a similar fashion. 

For detailed instructions about all 
aspects of the database, see appendix 
A of Myint-U et al. (2008, revised 
2009) at http://ies.ed.gov/ncee/edlabs/ 
projects/project.asp?ProjectID=37. 



dropout rates. They are also investigating how the 
strategies identified in the Dropout Prevention 
Practice Guide (Dynarski et al. 2008) can inform 
future data collection and reporting. 



STUDY LIMITATIONS 

This study has several limitations. The complete- 
ness and accuracy of the data depend on the 
diligence of key respondents. Several measures 
involving district staff were designed to in- 
crease the likelihood that the study would yield 



comprehensive data about dropout prevention 

programs in the nine districts: 

• Identifying staff most directly responsible for 
dropout prevention. 

• Providing them with all questions in advance. 

• Interviewing them carefully. 

• Examining their responses for internal consis- 
tency and agreement with data obtained from 
other sources. 
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• Recontacting them to probe for missing or 
inconsistent data. 

• Securing their assurance that collected data 
were accurate. 

Nevertheless, it is possible that interviews of other 
staff would have produced different data. 

Not all districts had a strict definition of dropout 
program. The list of core strategies provided to 
respondents was intended to encourage reporting 
of all programs that could reasonably be consid- 
ered to have dropout prevention as their aim. Still, 
some districts may have reported programs that 
other districts would not have characterized as 
dropout prevention programs. (For example, one 
district might consider a safe schools program a 
component of its dropout prevention efforts, while 
another district might not.) 



Some district respondents did not report all of 
the information sought about their programs. 
For example, implementation costs were not 
reported for 20 programs, demographic data 
for 6 programs, and funding information for 
4 programs. 

Because the research team used the National 
Center for Education Statistics definition of mid- 
size city, no Maryland or District of Columbia 
school district was included in the study and large 
districts were excluded. 

By focusing on districts with the highest drop- 
out rates, the study excluded socioeconomically 
similar districts with lower dropout rates, whose 
dropout prevention programs could be of consid- 
erable interest. 
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APPENDIX A 2008, p. D-1).^ No Maryland or District of Columbia 

STUDY METHODOLOGY school districts served cities that met this definition. 



This report replicated the Regional Educational 
Laboratory (REL) Northeast and Islands study in 
sample selection and data collection. 



Selecting the sample 

To add comparable data to the REL Northeast and 
Islands database, this study focused on school dis- 
tricts serving mid-size cities, so that “staff would 
be able to collect comprehensive information 
on programs and policies implemented within 
the whole district, rather than just in individual 
schools” (Myint-U et al. 2008, revised 2009, p. 5). 
Thus, the largest districts were excluded from data 
collection. 

First, a list of districts serving mid-size cities in 
each Mid- Atlantic state was compiled using the 
National Center for Education Statistics (NCES) 
definition of mid-size city as a “central city” of a 
metropolitan area “having a population less than 
250,000” (National Center for Education Statistics 



Next, the districts were merged into a single list 
and ranked from high to low by cumulative four- 
year dropout rates (the percentage of students 
who were enrolled in grade 9 but did not gradu- 
ate four years later), percentage of racial/ethnic 
minority students, and percentage of students 
from households living below the poverty line. 
Districts that ranked below 18 in any factor were 
excluded to concentrate on districts with the great- 
est challenges. 

The rankings for each factor were then summed 
to form a cumulative rank, and the nine highest 
ranking districts were selected. Three of the nine 
failed to respond to invitations to participate in 
the project and were replaced by the three next 
highest ranking districts. The nine participating 
districts served communities with populations 
of 24,742-107,250 (as of July 2008). Because the 
selection process focused on districts with specific 
demographic characteristics, it might not be 
representative of all districts in the Mid-Atlantic 



TABLE Al 

Key characteristics of participating Mid-Atlantic Region school districts, 2008 



State 


District 


National Center for 
Education Statistics 
district locale code 
(post-2006)^ 


Percentage of 
racial/ethnic 
minority students 
(2005/06) 


Percentage of 
students in grade 9 
in 2003/04 who 
did not graduate 
in 2006/07 


Percentage of 
students from 
households with 
low socioeconomic 
status (2004) 


Delaware 


Capital (Dover) 


City: Small 


58 


49 


17 


New Jersey 


Bridgeton 


Town: Fringe 


87 


37 


25 


Pennsylvania 


Allentown City 


City: Midsize 


76 


38 


29 


Pennsylvania 


Harrisburg City 


City: Small 


95 


45 


34 


Pennsylvania 


Lancaster 


City: Small 


79 


35 


28 


Pennsylvania 


Lebanon 


City: Small 


49 


37 


26 


Pennsylvania 


Reading 


City: Small 


85 


53 


38 


Pennsylvania 


Williamsport Area 


City: Small 


27 


31 


21 


Pennsylvania 


York City 


City: Small 


80 


41 


40 



a. All districts were categorized as mid-size cities in the pre-2006 locale code system. 

Source.- Authors' analysis based on data from U.S. Department of Education (2007a, 2007b, and 2009); www.doe.k12.de.us/reports_data/enrollment/default. 
shtml; www.doe.k12.de.us/reports_data/gradrate/default.shtml; www.state.nj.us/education/data/; and www.education.state.pa.us/portal/server.pt/ 
community/data_and_statistics/7202. 
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Region or of any jurisdiction in the Mid- Atlantic 
Region. Table A1 displays key characteristics of 
the districts. 



Collecting the data 

Supporting materials, based on materials used in 
the REL Northeast and Islands study, included 
introductory letters to school personnel (appen- 
dix C), model commitment letters for district use 
(appendix D) interview guides (appendix E), and a 
template for recording information (appendix F).^ 
The template was developed directly from the REL 
Northeast and Islands database fields to ensure 
that Mid- Atlantic Region data could be integrated. 
It provided respondents with lists of response 
categories for each question, so qualitative data did 
not have to be coded. 

Next, interview questions were developed on 
programs and policies focusing on dropout 
prevention, dropout reduction, school comple- 
tion, and students at a higher risk of dropping 
out, such as pregnant teens, students who are 
old for their grade, and students with emotional 
or behavioral challenges (Dynarski and Gleason 
2002; Neild and Balfanz 2006; Osher, Morrison, 
and Bailey 2003; Rumberger 2001). The NCES 
event dropout rate was used to define dropout: 
“students of any grade who leave school between 
the beginning of one school year and the begin- 
ning of the next without earning a high school 
diploma or its equivalent” (Laird, DeBell, and 
Chapman 2006). 

Before contacting the districts, an Internet search 
was conducted focusing on school districts, state 
departments of education, local media, and 
community-based organizations (such as the 
Spanish American Civic Association of Lancaster 
or the Olivet Boys and Girls Club of Reading and 
Berks County). These sources provided prelimi- 
nary information about local dropout problems 
and what districts and states were doing to address 
them. The information, used to build knowledge 
about local conditions, facilitated discussion with 
district stalf. 



Initial contact was made in phone calls and emails 
to district administrative offices. The research 
team explained the project, answered questions, 
and asked to be connected to the person most 
directly responsible for dropout prevention efforts. 
In one district, for example, this staff member was 
the coordinator of counseling and dropout preven- 
tion programs, in another, the executive director 
of community and student services, and in a third, 
the principal. 

Respondents received explanatory materi- 
als, a model for a commitment letter from the 
superintendent, and interview consent forms. 

The model commitment letter (see appendix 
D) stipulated that only descriptive information 
would be collected, that the superintendent or a 
designee could review the information and delete 
anything considered inappropriate for inclu- 
sion, and that neither database nor publication 
would identify any district staff. Respondents 
agreed to secure district commitment letters, 
complete consent forms, and gather informa- 
tion on district dropout prevention programs 
and policies by consulting their files and those 
of colleagues. Commitment letters, signed by the 
superintendent or a designee, were received for 
each district, and consent forms were received for 
each respondent. 

Respondents then received a template for record- 
ing data (see appendix F) several days before 
scheduled interviews so they could review the 
questions and prepare by consulting colleagues 
or compiling information. Respondents were 
asked to review the core strategies described in 
the template to guide them on what programs to 
report. 

Each respondent was interviewed by phone for 
about 15-30 minutes and asked to send additional 
materials describing the programs. 

After each interview, the research team looked for 
discrepancies (contradictions involving funding, 
numbers of students served, program start and 
end dates), in interview data, written materials. 
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and data obtained from websites and contacted 
respondents for clarification. Twice where respon- 
dents failed to report programs mentioned on 
websites, the research team probed for additional 
information; in both cases, respondents reported 
that the websites were out of date. Finally, each re- 
spondent received a complete data file to confirm 
its accuracy and verify that the superintendent 
or a designee had authorized its inclusion in the 
database. 

All project data, including written notes and 
supplemental materials, were stored at Rutgers 
University, in accordance with the requirements of 
Rutgers’ Institutional Review Board for the Protec- 
tion of Human Subjects. 



Adding to the database 

Data from this study were entered into a blank copy 
of the REL Northeast and Islands database of drop- 
out prevention programs. This copy was then for- 
warded to REL Northeast and Islands for integration 
into a combined database titled Database of Drop- 
out Prevention Programs in Selected Low-Income 
School Districts in the Northeast and Islands Region 
and the Mid- Atlantic Region. The database is main- 
tained by REL Northeast and Islands or its designee, 
but other regional educational laboratories, states, 
and organizations may be authorized to add data, as 
appropriate. REL Northeast and Islands makes any 
decisions on updating database fields. The database 
is updated and expanded periodically. 
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APPENDIX B 

DROPOUT PREVENTION INTERVENTIONS 
EVALUATED BY THE WHAT WORKS 
CLEARINGHOUSE (AS OF MAY 1, 2010) 

What Works Clearinghouse reviews of dropout 
interventions focus on three outcome domains: 
staying in school, progressing in school, and com- 
pleting school. Interventions are assigned to one of 
six categories for each domain: 

• Positive effect. Strong evidence of a positive 
effect with no overriding contrary evidence. 

• Potentially positive effect. Evidence of a positive 
effect with no overriding contrary evidence. 

• Mixed effect. Evidence of inconsistent effects. 

• No discernible effect. No affirmative evidence 
of an effect. 

• Potentially negative effect. Evidence of a 
negative effect with no overriding contrary 
evidence. 

• Negative effect. Strong evidence of a negative 
effect with no overriding contrary evidence. 

Interventions with positive or potentially positive 
effects on two domains were: 

• Accelerated Middle Schools: staying in school 
(potentially positive) and progressing in 
school (positive). 

• ALAS (Achievement for Latinos through 
Academic Success): staying in school and pro- 
gressing in school (potentially positive). 

• Career Academies: staying in school and pro- 
gressing in school (potentially positive). 



• Check & Connect: staying in school (positive) 
and progressing in school (potentially positive). 

Interventions with potentially positive effects on 
one domain were: 

• Financial Incentives for Teen Parents to Stay 
in School: staying in school. 

• High School Redirection: progressing in 
school. 

• Job Corps: completing school. 

• JOBSTART: completing school. 

• New Chance: completing school. 

• Talent Development High Schools: progress- 
ing in school. 

• Talent Search: completing school. 

• Twelve Together: staying in school. 

Interventions with no discernible effects on any 
domain were: 

• First Things First 

• Middle College High School 

• Project Grad 

• Quantum Opportunity Program 

• Summer Training and Education Program 
(STEP) 

None of the identified programs was found to have 
mixed effects, a potentially negative effect, or a 
negative effect. 
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APPENDIX C 
INTRODUCTORY LETTER 

Dear [Superintendent]: 

The Regional Educational Laboratory Mid- Atlantic, funded by the U.S. Department of Education’s Institute for 
Education Sciences, is helping to build a database of current dropout prevention policies and programs that target 
urban students throughout the country (see enclosed news release about the project’s initial phase, concentrating 
on New England districts). This database is intended to (1) support collaboration across districts involving promis- 
ing dropout prevention strategies and (2) document local efforts to use evidence-based programs and strategies, 
such as those identified by the U.S. Department of Education’s What Works Clearinghouse. 

We plan to collect information this year on nine mid-size city school districts in our region. Your district was cho- 
sen as one of the initial participants because of its size, the diversity of its student population, and your presumed 
interest in addressing dropout issues. 

We hope to acquire necessary information in brief telephone interviews (15 to 30 minutes) with you or other staff 
you may recommend. To minimize your time investment, we will first search the Web for publicly available infor- 
mation on dropout programs in your district and send our questions to you in advance. 

Information collected will be at the district level and will not include any personal information. When we have 
finished collecting information from your district, we will submit it to you for review and will delete any data that 
you don’t want to include in the database. 

Within a few days, we will telephone you to answer any questions you may have about the project and to schedule a 
convenient time for you to participate in a brief telephone interview. 

Thank you in advance for your time and consideration. 



Sincerely, 

Jennifer Bausmith, Ph.D. 
Co -Principal Investigator 
Rutgers University 



Enel: REL-NEI news release 
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APPENDIX D 

MODEL COMMITMENT LETTER 

Jennifer Bausmith, Ph.D. 

Regional Educational Laboratory Mid-Atlantic 
Center for Effective School Practices 
Rutgers, The State University of New Jersey 
80 Cottontail Lane 
Somerset, NJ 08873 

Dear Dr. Bausmith: 

This letter serves as a commitment for our district to collaborate with the Regional Educational Laboratory Mid- 
Atlantic in a study of dropout prevention programs in the region. 

I understand that the intent of the study is to develop a database providing useful and relevant information for this 
district and other mid-sized urban school districts nationally. The research team will also produce a short publica- 
tion summarizing the methods and findings of the study. 

I also understand that: 

• The study will take the form of brief telephone interviews with key district personnel, who will receive a com- 
plete list of questions in advance; 

• Information collected will be restricted to descriptive information about district dropout programs; 

• I will have an opportunity to review all information collected by the researchers and delete anything I think 
should not be included in the database; and 

• Neither database nor publication will identify any interviewees. 



Sincerely, 



APPENDIX E. INTERVIEW GUIDE 



17 



APPENDIX E 
INTERVIEW GUIDE 

1. Introduce the purpose of the phone interview: 
“Thank you for agreeing to speak with me about 
the kinds of dropout prevention policies and 
programs that are being utilized in the [name of 
school district] school district. The purpose of this 
project is to collect information on how schools in 
the Mid-Atlantic Region are addressing the issue 
of high school dropout. 

Before we start, do you have any questions I can 
answer?” 

2. Ask for a district-level letter of commitment and 
a signed consent form from the interviewee. State 
that the information will be used at the district level 
with no personal information collected or reported. 
The data will be used for writing final reports and 



for populating the existing database developed by 
Regional Educational Laboratory Northeast and 
Islands. State that the interview process should not 
take more than 60 minutes of their time. 

3. Conduct the interview — referring to the inter- 
viewer template for recording data. For each 
program mentioned, try to get answers to every 
question on the template to enter into the project 
data file. 

4. Request written materials (by mail, fax, or email) 
that may be available on those dropout prevention 
programs and policies mentioned and that can 
help get information for the database. 

5. Once the interview is complete, request contact 
information for the appropriate individual in the 
district who can review the completed data files to 
ensure accuracy of data entered. 
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APPENDIX F 

INTERVIEW TEMPLATE FOR RECORDING DATA 

What are the district’s 2008/09 student 
demographics? 

Student population: 

% limited English proficient: 

% free or reduced-price lunch: 

Student ethnicity: 

% Asian 

% Black 

% Latino 

% Native American 

% White 

% Other 

Program/policy title: 



Names of schools implementing the program/policy 



Intervention level (check all that apply) 

□ Entire student population 

□ Students considered at risk because 
they belong to some subset of the school 
population 

□ Students considered at risk because of 
individual performance or behaviors 

Target populations (check all that apply) 

□ Students with academic needs 

□ Students with limited English proficiency 

□ Students who would be part of the first 
generation in their family to attend 
college 

□ Students in low socioeconomic status 
families 

□ Students who are pregnant or mothers 

□ Students returning from incarceration 

□ Special needs students with behavioral 
challenges 

□ Special needs students with learning 
disabilities 

□ Special needs students with mental health 
needs 

□ Students who are chronically truant or 
absent 



Brief description (1-3 sentences on program/policy 
and target population; include website address, if 
applicable) 



Target grades (circle all that apply) 
6 7 8 9 10 11 12 

Number of participants 

2009/10 

2008/09 

2007/08 



Demographics of participants 

% Male 

% Female 

% Asian 

% Black 

% Latino 

% Native American 

% White 

Project/policy goals (check all that apply) 

□ Address behavioral needs 

□ Address school safety and environment 

□ Decrease truancy 

□ Improve academic performance 

□ Increase school attachment 

□ Promote college planning and linkages 

□ Provide career planning and preparation 

□ Provide mental health support 

□ Provide support during transitions 
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Core strategies (check all that apply) 





Strategy 


Description I 


□ 


Accelerated credit accumulation 


Provides students with opportunities to fulfill credits in an expedited way so 
they can "catch up" with their same-age peers. 


□ 


Advocating for student needs 


Encourages program staff to communicate with school officials or key personnel 
about students' needs and ways to address them. 


□ 


Career education and workforce 
readiness 


Introduces and exposes students to different types of careers and/or provides 
skills for entering the workforce. 


□ 


Case management/service 
coordination 


Provides students or families who require multiple services with coordinated 
care throughout the service delivery process. 


□ 


Community collaboration 


Works with various community agencies and individuals to increase school- 
community collaboration and to link students to services. 


□ 


Engaging and supporting 
families 


Involves parents, guardians, and other family members in program activities; 
provides support to families to help them address issues that may facilitate 
dropout. 


□ 


Individualized or culturally/ 
linguistically relevant instruction 


Customizes instruction to match students' needs and abilities and recognizes 
and incorporates cultural and linguistic diversity of students. 


□ 


Instructional technologies 


Uses innovative new technologies, such as teacher-supported computer-based 
learning, to increase student motivation. 


□ 


Mentoring 


Matches students with adult mentors in an effort to establish a close, supportive 
one-on-one relationship. 


□ 


Monitoring attendance 


Uses tools or strategies to help schools more closely monitor whether a student 
is in school and to contact parents/guardians to let them know their child is 
absent. 


□ 


Out-of-school enrichment 


Provides students with after-school, Saturday, and summer enrichment 
programs. 


□ 


Professional development 


Provides opportunities for teaching staff to gain skills they can use in and 
outside the classroom to enrich their own experiences and the experiences of 
their students. 


□ 


Social and emotional learning 
curricula 


Uses curricula in classrooms to help students develop social and emotional 
learning skills (e.g., conflict resolution) to deal with issues that may place them 
at risk for dropping out. 


□ 


Social and emotional transition 
support 


Focuses on providing support to students who are in "transition" periods (e.g., 
transition from middle to high school, pregnancy, returning from incarceration, 
newly immigrated, parenthood); also includes provision of support to students 
with mental health needs. 


□ 


Systemic/policy renewal 


Focuses on creating a new or renewed coordinated district-level policy related 
to dropout prevention that will address most current issues and risks. 


□ 


Transforming the school 
environment 


Strives to create an overall school environment that is caring, safe, and 
emotionally supportive and in which students feel safe and a sense of respect 
and self-worth; may include smaller communities. 


□ 


Tutoring/extra classes 


Provides students with extra academic support for subject matters in which 
they are not excelling or are failing. 



In-school staff involvement (check all that apply) 


□ 


Dropout specialist 


□ Adjustment counselor 


□ 


Grant writer 


□ Behavioral resource office 


□ 


Guidance counselor 


□ Crisis counselor 


□ 


Nutrition counselor 


□ Daycare providers 


□ 


Paraprofessional 


□ Director 


□ 


Parent- community liaison 
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□ Principal/administrator 

□ Psychologist/therapist 

□ Resource officer 

□ School nurse 

□ Schoolwide 

□ Security guard 

□ Social worker 

□ Special education staff 

□ Support specialist 

□ Student advocate 

□ Teacher 

□ Truancy/attendance officer 

□ Tutor 

Out-of-school staff involvement (check all that 
apply) 

□ Americorps 

□ CBO [Community-based organization] 

□ Corporate 

□ Department of youth services or social 
services 

□ Health clinic 

□ Job Corps 

□ Local higher education 

□ Mental health services 

□ Mentoring program 

□ Parents 

□ Police 

□ Religious affiliates 



□ Truancy court 

□ Tutor 

□ Other volunteer 

Funding sources (check all that apply) 

□ District 

□ State 

□ Federal 

□ Private 

Approximate cost to implement in 2008109: 



Start date: 

End date (if applicable): 

Reason for discontinuation (if applicable): 



Notes re: discontinuation (if applicable): 



NOTES 
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NOTES 



The authors are indebted to staff of Regional 
Educational Laboratory Northeast and Islands, 
particularly Athi Myint-U, for sharing research 
protocols and advice. 

1 . “Strong refers to consistent and generalizable 
evidence that a dropout prevention programs 
[sic] causes better outcomes. Moderate refers 
either to evidence from studies that allow 
strong causal conclusions but cannot be 
generalized with assurance to the popula- 
tion on which a recommendation is focused 
(perhaps because the findings have not been 
widely replicated) or to evidence from studies 
that are generalizable but have more causal 
ambiguity than offered by experimental 
designs (e.g., statistical models of correla- 
tional data or group comparison designs for 
which equivalence of the groups at pretest is 
uncertain). Low refers to expert opinion based 
on reasonable extrapolations from research 
and theory on other topics and evidence from 
studies that do not meet the standards for 
moderate or strong evidence.” (Dynarski et al. 
2008, pp. 1, 3). 

2. The research team’s list of service goals was 
developed by REL Northeast and Islands staff, 
who made a master list of goals identified 

in programs reviewed by the What Works 
Clearinghouse and in the research literature 
(Bridgeland et al. 2006; Dynarski and Gleason 
2002; Neild and Balfanz 2006; Rumberger 



2001). They then consolidated goals where 
there was overlap (for example, consolidating 
“increasing school attachment through family 
support” and “increasing student attachment 
through mentoring” into the service goal “in- 
creasing school attachment”) and submitted 
their consolidated list to expert reviewers who 
confirmed its face validity. 

3. What Works Clearinghouse reviews are 
restricted to programs that have been the 
subject of studies that meet high evidence 
standards; the fact that an intervention lacks 
such studies is not an indication that it is 
ineffective. 

4. In March 2006, the NCES changed its system 
of district locale codes. Seven of the districts 
selected for the study based on 2004/05 locale 
codes (the most recent available at the time) 
were assigned a new categorization of “City: 
Small;” one changed to “Town: fringe;” and 
one retained the designation of “City: Mid- 
size” (see table Al). 

5. The research team confirmed the determina- 
tion of REL Northeast and Islands research- 
ers that all information to be collected was 
publicly available on websites or from districts 
upon request and ensured in introductory 
and district commitment letters that district 
administrators would be asked to authorize 
release of all information before dissemina- 
tion/publication. These authorizations were 
attained. 
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